{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用字符级RNN分类名称\n",
    "\n",
    "[![](https://gitee.com/mindspore/docs/raw/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/tutorials/source_zh_cn/middleclass/text/rnn_classification.ipynb)\n",
    "\n",
    "循环神经网络（Recurrent Neural Network, RNN）是一类以序列（sequence）数据为输入，在序列的演进方向进行递归（recursion）且所有节点（循环单元）按链式连接的递归神经网络（recursive neural network），常用于NLP领域当中来解决序列化数据的建模问题。\n",
    "\n",
    "本教程为“从零开始的NLP”第一篇(!!!这是pytroch不是mindspore，去掉)，本教程我们将建立和训练基本的字符级RNN模型对单词进行分类，以帮助同学们理解循环神经网络原理。实验中，我们将训练来自18种语言的数千种姓氏，并根据拼写方式预测名称的来源：\n",
    "\n",
    "字符级RNN将单词作为一系列字符来读取，在每个步骤输出预测，隐藏此时的状态并输入到下一步中。 我们将最终的预测作为输出，该输出用来表示单词属于哪个类别(!!!整句话都不知道说的什么鬼。删掉吧"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(!!!加注释，-0.47好还是-3.57好？，这行代码执行代表什么？\n",
    "\n",
    "```bash\n",
    "$ python predict.py Hinton\n",
    "(-0.47) Scottish\n",
    "(-1.52) English\n",
    "(-3.57) Irish\n",
    "```\n",
    "\n",
    "(!!!加注释，-0.47好还是-3.57好？，这行代码执行代表什么？\n",
    "\n",
    "\n",
    "```\n",
    "$ python predict.py Schmidhuber\n",
    "(-0.19) German\n",
    "(-2.48) Czech\n",
    "(-2.68) Dutch\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 环境配置"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用`PyNative`模式运行实验，使用GPU环境。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore import context\n",
    "context.set_context(mode=context.PYNATIVE_MODE, device_target=\"GPU\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备数据\n",
    "\n",
    "点击[这里](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/middleclass/data.zip)下载数据，并将其提取到当前目录。(!!!数据集哪里来的，数据集是什么，介绍一下)\n",
    "\n",
    "可在Jupyter Notebook中执行以下代码完成数据集的下载，并将数据集解压完成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -N https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/middleclass/data.zip\n",
    "!unzip -n data.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`data/names`目录中包含 18 个文本文件，名称为`[Language].txt`。 每个文件包含一系列名称，每行一个名称。\n",
    "\n",
    "值得注意的是，数据的名字大多数是罗马音，需要将其从Unicode转换为ASCII。转换之后将得到一个字典，其中列出了每种语言的名称列表`{language: [names ...]}`。其中，通用变量(!!!什么是通用变量？)为“category”和“line”，代表语言种类及名称。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['data/names/Japanese.txt', 'data/names/Chinese.txt', 'data/names/Polish.txt', 'data/names/Scottish.txt', 'data/names/Vietnamese.txt', 'data/names/Czech.txt', 'data/names/Portuguese.txt', 'data/names/Dutch.txt', 'data/names/Greek.txt', 'data/names/French.txt', 'data/names/Arabic.txt', 'data/names/Russian.txt', 'data/names/Spanish.txt', 'data/names/Korean.txt', 'data/names/English.txt', 'data/names/Italian.txt', 'data/names/German.txt', 'data/names/Irish.txt']\n",
      "Slusarski\n"
     ]
    }
   ],
   "source": [
    "from io import open\n",
    "import glob\n",
    "import os\n",
    "import warnings\n",
    "import unicodedata\n",
    "import string\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "def findFiles(path): return glob.glob(path)\n",
    "\n",
    "print(findFiles('data/names/*.txt'))\n",
    "\n",
    "all_letters = string.ascii_letters + \" .,;'\"\n",
    "n_letters = len(all_letters)\n",
    "\n",
    "(!!!拆分出来，加中文注释)\n",
    "\n",
    "# Unicode转化为ASCII\n",
    "def unicodeToAscii(s):\n",
    "    return ''.join(\n",
    "        c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn'\n",
    "        and c in all_letters\n",
    "    )\n",
    "\n",
    "print(unicodeToAscii('Ślusàrski'))\n",
    "\n",
    "# 建立category_lines dictionary\n",
    "category_lines = {}\n",
    "all_categories = []\n",
    "\n",
    "(!!!拆分出来，加中文注释)\n",
    "\n",
    "# 读文件并且分列\n",
    "def readLines(filename):\n",
    "    lines = open(filename, encoding='utf-8').read().strip().split('\\n')\n",
    "    return [unicodeToAscii(line) for line in lines]\n",
    "\n",
    "for filename in findFiles('data/names/*.txt'):\n",
    "    category = os.path.splitext(os.path.basename(filename))[0]\n",
    "    all_categories.append(category)\n",
    "    lines = readLines(filename)\n",
    "    category_lines[category] = lines\n",
    "\n",
    "n_categories = len(all_categories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们有了`category_lines`字典，可以将每个`category`（语言种类）映射到`line`（名称）。在这个过程中同步建立了语言列表`all_categories`和语言数量`n_categories`，用于后续的处理。(!!!解释下面那行代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Abandonato', 'Abatangelo', 'Abatantuono', 'Abate', 'Abategiovanni']\n"
     ]
    }
   ],
   "source": [
    "print(category_lines['Italian'][:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 将名称转换为向量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(!!!为什么需要把name转变为向量？）现在需要将名称转变为向量。\n",
    "\n",
    "为了表示单个字母，我们使用大小为`<1 x n_letters>`的one-hot向量。(!!!介绍one-hot的作用\n",
    "\n",
    "> one-hot向量用0填充，但当前字母的索引处的数字为1，例如 `\"b\" = <0 1 0 0 0 ...>`。\n",
    "\n",
    "为了组成单词，我们将其中的一些向量连接成2D矩阵`<line_length x 1 x n_letters>`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      "  0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      "  0. 0. 0. 0. 0. 0. 0. 0. 0.]]\n",
      "(5, 1, 57)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from mindspore import Tensor\n",
    "from mindspore import dtype as mstype\n",
    "\n",
    "# 查找字母索引，例如\"a\" = 0\n",
    "def letterToIndex(letter):\n",
    "    return all_letters.find(letter)\n",
    "\n",
    "# 将字母变为 <1 x n_letters> Tensor\n",
    "def letterToTensor(letter):\n",
    "    tensor = Tensor(np.zeros((1, n_letters)),mstype.float32)\n",
    "    tensor[0,letterToIndex(letter)] = 1.0\n",
    "    return tensor\n",
    "\n",
    "# 将一行转化为<line_length x 1 x n_letters>，或者array形式的one-hot向量\n",
    "def lineToTensor(line):\n",
    "    tensor = Tensor(np.zeros((len(line), 1, n_letters)),mstype.float32)\n",
    "    for li, letter in enumerate(line):\n",
    "        tensor[li,0,letterToIndex(letter)] = 1.0\n",
    "    return tensor\n",
    "\n",
    "print(letterToTensor('J'))\n",
    "print(lineToTensor('Jones').shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 创建网络\n",
    "\n",
    "创建的RNN网络只有两个线性层(!!!图文对应，线性层是哪个名字？），它们在输入和隐藏状态(!!!这两个状态又是对应图中那个？）下执行，在线性层(!!!解释图）的输出之后是`LogSoftmax`层。其中，网络结构如图所示。\n",
    "\n",
    "![rnn](images/run1.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mindspore.nn as nn\n",
    "import mindspore.ops as ops\n",
    "class RNN(nn.Cell):\n",
    "    \n",
    "    def __init__(self, input_size, hidden_size, output_size):\n",
    "        super(RNN, self).__init__()\n",
    "        self.hidden_size = hidden_size\n",
    "        self.i2h = nn.Dense(input_size + hidden_size, hidden_size)\n",
    "        self.i2o = nn.Dense(input_size + hidden_size, output_size)\n",
    "        self.softmax = nn.LogSoftmax(axis=1)\n",
    "\n",
    "    def construct(self, input, hidden):\n",
    "        op = ops.Concat(axis=1)\n",
    "        combined = op((input, hidden))\n",
    "        hidden = self.i2h(combined)\n",
    "        output = self.i2o(combined)\n",
    "        output = self.softmax(output)\n",
    "        return output, hidden\n",
    "\n",
    "    def initHidden(self):\n",
    "        return Tensor(np.zeros((1, self.hidden_size)),mstype.float32)\n",
    "\n",
    "       \n",
    "n_hidden = 128\n",
    "rnn = RNN(n_letters, n_hidden, n_categories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在运行网络之前，我们需要输入代表当前字母的向量，以及先前的隐藏状态(!!!先前的是指代什么）。这里将隐藏状态初始化为0。在运行过程中，可以返回属于每种语言的概率和下一个隐藏状态(!!!下一个是哪个，为什么会有下一个）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "input = letterToTensor('A')\n",
    "hidden =Tensor(np.zeros((1, n_hidden)),mstype.float32)\n",
    "\n",
    "output, next_hidden = rnn(input, hidden)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了提高效率，避免在每一步中都创建一个新向量，因此将使用`lineToTensor`而不是`letterToTensor`，同时采取切片操作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-2.8951542 -2.8894851 -2.89519   -2.9023316 -2.88667   -2.8859577\n",
      "  -2.8895268 -2.8953846 -2.8703299 -2.8869376 -2.8837688 -2.8951073\n",
      "  -2.8964722 -2.898948  -2.8797808 -2.8864856 -2.885134  -2.9046268]]\n"
     ]
    }
   ],
   "source": [
    "input = lineToTensor('Albert')\n",
    "hidden = Tensor(np.zeros((1, n_hidden)),mstype.float32)\n",
    "\n",
    "output, next_hidden = rnn(input[0], hidden)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，输出为`<1 x n_categories>`形式的向量，其中每个数字都代表了分类的可能性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练\n",
    "\n",
    "### 准备训练\n",
    "\n",
    "在接受训练之前(!!!太英文)，需要一些前处理工作。这里使用`ops.TopK`获得网络模型输出的最大值，也就是最输出分类类别的概率(!!!用概率别用可能性）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Greek', 8)\n"
     ]
    }
   ],
   "source": [
    "def categoryFromOutput(output):\n",
    "    topk = ops.TopK(sorted=True)\n",
    "    top_n, top_i = topk(output,1)\n",
    "    category_i = top_i.asnumpy().item(0)\n",
    "    return all_categories[category_i], category_i\n",
    "\n",
    "print(categoryFromOutput(output))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们还将希望有一种快速的方法来获取训练示例，包括名称及其所属的语言类别。(!!!这句话改为人说的话）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "category = Russian / line = Avdulov\n",
      "category = Spanish / line = Puerta\n",
      "category = Polish / line = Kowalczyk\n",
      "category = Arabic / line = Hakimi\n",
      "category = Scottish / line = Miller\n",
      "category = Vietnamese / line = Thai\n",
      "category = Czech / line = Kacirek\n",
      "category = Portuguese / line = Ferreiro\n",
      "category = Vietnamese / line = Do\n",
      "category = Polish / line = Fabian\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "def randomChoice(l):\n",
    "    return l[random.randint(0, len(l) - 1)]\n",
    "\n",
    "def randomTrainingExample():\n",
    "    category = randomChoice(all_categories)\n",
    "    line = randomChoice(category_lines[category])\n",
    "    category_tensor = Tensor([all_categories.index(category)], mstype.int32)\n",
    "    line_tensor = lineToTensor(line)\n",
    "    return category, line, category_tensor, line_tensor\n",
    "\n",
    "for i in range(10):\n",
    "    category, line, category_tensor, line_tensor = randomTrainingExample()\n",
    "    print('category =', category, '/ line =', line)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练网络\n",
    "\n",
    "可以通过大量数据来进行网络训练(!!!还可以通过其他方法吗？，改为人话）。\n",
    "\n",
    "由于网络的最后一层是`nn.LogSoftmax`，所以采用`NLLLoss`来作为损失函数。(!!!为什么，我不知道，解释清楚）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mindspore.ops as ops\n",
    "from mindspore.nn.loss import Loss\n",
    "\n",
    "class NLLLoss(Loss):\n",
    "    def __init__(self, reduction='mean'):\n",
    "        super(NLLLoss, self).__init__(reduction)\n",
    "        self.one_hot = ops.OneHot()\n",
    "        self.reduce_sum = ops.ReduceSum()\n",
    "    def construct(self, logits, label):\n",
    "        label_one_hot = self.one_hot(label, ops.shape(logits)[-1], ops.ScalarToArray(1.0), ops.ScalarToArray(0.0))\n",
    "        loss = self.reduce_sum(-1.0 * logits * label_one_hot, (1,))\n",
    "        return self.get_loss(loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = NLLLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每个循环训练将会执行下面几个步骤：(!!!像这样用人话表达出来，别给粘英文翻译)\n",
    "\n",
    "- 创建输入和目标向量\n",
    "- 创建归零的初始隐藏状态(!!!归零是什么)\n",
    "- 学习每个字母并保存下一个字母的隐藏状态\n",
    "- 比较最终输出与目标值\n",
    "- 反向传播梯度变化\n",
    "- 返回输出和损失值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore import nn, ops, context\n",
    "\n",
    "class WithLossCellRnn(nn.Cell):\n",
    "    def __init__(self, backbone,loss_fn):\n",
    "        super(WithLossCellRnn, self).__init__(auto_prefix=True)\n",
    "        self._backbone = backbone\n",
    "        self._loss_fn = loss_fn\n",
    "\n",
    "    def construct(self, line_tensor, hidden, category_tensor):\n",
    "        for i in range(line_tensor.shape[0]):\n",
    "            output, hidden = self._backbone(line_tensor[i], hidden)\n",
    "        return self._loss_fn(output, category_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "(!!!加中文注释)\n",
    "\n",
    "from mindspore import ParameterTuple\n",
    "rnn_cf = RNN(n_letters, n_hidden, n_categories)\n",
    "optimizer = nn.Momentum(filter(lambda x:x.requires_grad,rnn_cf.get_parameters()),0.001,0.9)\n",
    "net_with_criterion = WithLossCellRnn(rnn_cf, criterion)\n",
    "net = nn.TrainOneStepCell(net_with_criterion, optimizer)\n",
    "net.set_train()\n",
    "def train(category_tensor, line_tensor):\n",
    "    hidden = rnn_cf.initHidden()\n",
    "    loss = net(line_tensor,hidden,category_tensor)\n",
    "    for i in range(line_tensor.shape[0]):\n",
    "        output, hidden = rnn_cf(line_tensor[i], hidden)\n",
    "    return  output, loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面进行模型训练。通过`print_every`，每隔500个样本打印一次。通过✓、✗来表示每个节点上模型判断的正误。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "500 5% (0m 55s) 2.9511454 Mai / Italian ✗ (Vietnamese)\n",
      "1000 10% (1m 39s) 2.7946725 Timonnikov / Portuguese ✗ (Russian)\n",
      "1500 15% (2m 23s) 2.581205 Morandi / Polish ✗ (Italian)\n",
      "2000 20% (3m 4s) 2.7694151 Ceallachan / Vietnamese ✗ (Irish)\n",
      "2500 25% (3m 47s) 2.8895192 Bairamkuloff / Polish ✗ (Russian)\n",
      "3000 30% (4m 29s) 3.07459 Sarkozi / Italian ✗ (French)\n",
      "3500 35% (5m 10s) 2.8098087 Mcnab / Chinese ✗ (Irish)\n",
      "4000 40% (5m 53s) 2.1731405 Arlotti / Italian ✓\n",
      "4500 45% (6m 35s) 3.2535906 De laurentis / Greek ✗ (Italian)\n",
      "5000 50% (7m 19s) 2.6969473 O'Neal / Irish ✓\n",
      "5500 55% (7m 59s) 3.0052674 Fabian / Scottish ✗ (French)\n",
      "6000 60% (8m 42s) 2.9978802 Chi / Italian ✗ (Korean)\n",
      "6500 65% (9m 24s) 1.4152222 Hagias / Greek ✓\n",
      "7000 70% (10m 7s) 2.5804625 Malouf / Arabic ✓\n",
      "7500 75% (10m 48s) 2.615438 Reid / Scottish ✓\n",
      "8000 80% (11m 31s) 2.6654413 O'Boyle / French ✗ (Irish)\n",
      "8500 85% (12m 14s) 3.0208545 Wruck / Polish ✗ (German)\n",
      "9000 90% (12m 55s) 1.734647 Wojewodzki / Polish ✓\n",
      "9500 95% (13m 38s) 2.5977328 Daly / English ✗ (Irish)\n",
      "10000 100% (14m 20s) 2.6486962 Rodagh / Vietnamese ✗ (Irish)\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import math\n",
    "\n",
    "n_iters = 10000\n",
    "print_every = 500\n",
    "plot_every = 100\n",
    "\n",
    "# Keep track of losses for plotting\n",
    "current_loss = 0\n",
    "all_losses = []\n",
    "\n",
    "def timeSince(since):\n",
    "    now = time.time()\n",
    "    s = now - since\n",
    "    m = math.floor(s / 60)\n",
    "    s -= m * 60\n",
    "    return '%dm %ds' % (m, s)\n",
    "\n",
    "(!!!拆分出来，加中文注释)\n",
    "\n",
    "\n",
    "start = time.time()\n",
    "for iter in range(1, n_iters + 1):\n",
    "    category, line, category_tensor, line_tensor = randomTrainingExample()\n",
    "    output, loss = train(category_tensor, line_tensor)\n",
    "    current_loss += loss\n",
    "\n",
    "    # 根据迭代打印loss、对应的name和预测值\n",
    "    if iter % print_every == 0:\n",
    "        guess, guess_i = categoryFromOutput(output)\n",
    "        correct = '✓' if guess == category else '✗ (%s)' % category\n",
    "        (!!!下面这一行只有你自己看得懂)\n",
    "        print('%d %d%% (%s) %s %s / %s %s' % (iter, iter / n_iters * 100, timeSince(start), loss.asnumpy(), line, guess, correct))\n",
    "\n",
    "    # 将当前loss添加至all_losses\n",
    "    if iter % plot_every == 0:\n",
    "        all_losses.append((current_loss / plot_every).asnumpy())\n",
    "        current_loss = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制结果\n",
    "\n",
    "从`all_losses`绘制网络模型学习过程中每个step得到的损失值，可显示网络学习情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'all_losses' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-2ec1946169c3>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mall_losses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'all_losses' is not defined"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.ticker as ticker\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(all_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评估结果\n",
    "\n",
    "为了查看网络在不同分类上的表现，我们将创建一个混淆矩阵，行坐标为实际语言，列坐标为预测的语言。为了计算混淆矩阵，使用`evaluate()`函数通过RNN网络运行样本(!!!这句话不通)，这一过程相当于在`train()`训练过程中不进行反向传播(!!!那就是直接与推理，绕一圈说话的吗？)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVwAAAEvCAYAAAAJoHlDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABNe0lEQVR4nO2dd5hdVfW/308SSCABREKTFpAAAkKAgKCAQRFREPArSrNgoagU9adixYhdbDSliaCiKKAQBQSld9KBUARC0ACKkSIlhJTP74+9b3Jy57YzuTNzZ7Le5znP3LPPOnvvO3Nn3X3WXkW2CYIgCHqeQX09gSAIguWFULhBEAS9RCjcIAiCXiIUbhAEQS8RCjcIgqCXCIUbBEHQS4TCDYIg6CVC4QZBEPQSoXCXM5R4v6QT8/mGknbq63kFwfKAItJs+ULSz4BFwFtsv07S6sA1tnfs46kFwYBnSF9PIOh13mB7e0lTAWw/I2nFvp5UECwPhElh+WO+pMGAASStSVrxtg1JK0v6qqRz8vloSfu2c4wg6I+Ewl3+OBX4I7CWpG8BtwDfbvMYvwDmAbvk88eBb7Z5jCDod4QNdzlE0hbAWwEB19q+v839T7I9VtJU29vltum2t23nOEHQ34gV7nKGpNcCj9o+A7gXeJukV7V5mFckrcQSs8VrSSveIFiuCYW7/HEpsFDSpsBZwAbAb9o8xteAvwAbSLoQuBb4fJvHCIJ+R5gUljMkTcleCp8H5to+rfjo38Zx1gB2Jpkt7rA9p539B0F/JFa4yx/zJR0CfBD4c25boZ0DSHoT8LLtK4BXAV+StFE7xwiC/kgo3OWPD5O8B75l+1FJGwO/avMYPwNekrQt8BngEeCXbR4jCPodYVJYDskbWhvafrCH+q+YLU4EHrf980pbT4wXBP2FWOEuZ0h6FzCNtKmFpDGSJrR5mOclfRH4AHCFpEG02WwRBP2RULglGCARVOOBnYBnAWxPAzZp8xgHkdzAPmL7X8D6wMltHiMI+h2hcMsxECKo5tt+rqqtraG9WcleCgzNTXNI0W1BsFwTCrccr7X9fWA+gO2XSG5P/YkZkg4FBucV+mnAbe0cQNIRwCUkP1+A9YDL2jlGEPRHQuGWYyBEUB0LbEWa92+B/wGfavMYnwTelPvG9kPAWm0eIwj6HZGesRzVEVRvAg7v0xmVJK/Kv5yPnmKe7VektPiXNIT8JRUEyzOhcEtg+6+SprAkgur4/hZBJWks8CVgFIW/v+1t2jjMjZK+BKwk6W3AJ4A/tbH/IOiXhB9uCXIE1TTbL0p6P7A9cIrtx5rctysw2vYvcv7ZEbYfbeO8BgNrs7QC/Ucd2QeBzwH3UNgsa/YeSs5nEPBRYC/SF9PVwLmOD9uAocxnLlhCKNwSSLob2BbYhuSx8HPgfbbf3OCerwFjgc1tbybpNcDFtt/UpjkdSzJ1/JslCtT1VqySbrG9azvGDrpH/tI9gq5PGR/pqzmVoexnLlhCmBTKscC2Je0PnJEjqD7a5J53A9sBUwBsPyFplTbO6XiSMv9vi/Jfk3QuKYPX4g0/239o14QkPUoNm63tdvv7dgSSNiM9NWzE0gr0LXVuuRy4GfgbsLDHJ9h+yn7mgkwo3HIUI6h2azGC6pWspCueDcPbPKd/AtV+tY34MLAFad6LVydA2xQuaUVfYRjwXuDVbey/07gYOBM4h9YU6Mq2T+jZKfUoZT9zQSZMCiWQtA5wKDDR9s2SNgTG2a6bmEXSZ4HRwNuA7wAfAX5j+7RlnMtn8sutgM2BK1h6xfqjOvc9aHvzZRm7O0iabHuH3h63Nyj73iR9E7jN9pU9NJ+hwHvoarI4qU39/5wSn7lgCbHCLYHtf0m6lKRAoYUIKts/yDv1/yN9SE+0/dc2TKdilvhHPlbMRzNuk7Sl7fvaMIeaSComqRlEWvEO5M/anyR9gvRZKCqgp4tCkp4nPU2IlLJyHimIRkncq7ZpPpeTVqCT6Rk/8bKfuSATK9wS5AiqI4FX236tpNHAmbbf2uCe4aTcsAslbU5SulfZnt8D8xtE8oD4XwOZ+4HXAo+S/hkr/+xt2/CQdH3hdAEwC/hBT2Un62uyzboat8tmXXaTTdK9trdux9hBewmFWwJJ00iJX+4sFEe8x/brG9wzGdgNWJ1UIXcSya57WJvm9BvgaJLtcCKwKslVrWaymHqJwNvpFtaJSHojXRVWn+ToreNe+JMGrny3kTbZJlOwEdu+tI782cBptu9p++RZ/AXweZI5a1hhPvU2CYPMQH7M6wm6E0El2y9lb4af2f5+VtztYkvb/5N0GHAV8AXSP2ZNhVtRrJLWovDP0k4K9uUizwGTc3ayXkXSr0ir+mksUVimRlL0/EQy1/aifD4IGJYj9BqNsTWwJUsroHoK/WfAtjlB+/8DziUlga/nXtjSJpuke0jvawjwYUkz6ZmnmAuB3wH7kr7sPwT8p019D2hC4ZajOxFUkrQLcBgpGABgcBvntIKkFYADgNNtz694RNSZzH7AD4HXAE+RXJnuJ61W2sXYfFR+N/sCdwNHS7o4JwDqTcaSvphaeZy7FtgTeCGfrwxcA7yx3g3Z13ocSeFeCbyD9DRTT+EW3QtPb8G98M+S3tnCJtsypQqVtB5dXdtuqiG6Rp7z8bZvJP1fTFyWsZcbbMfR4kHaADqC5AZ0SX6tJve8GZgAnJDPNwFObeOcjiOlibyStJLZCLi5gfx0YA1gaj7fA/h5kzEGkxT0hpWjifxNJFty5XwEcCOwEnBfG97zxjXadmwgfzGwbot9T2ulrer6PfmzMT2frw38tYH8jcAXgb8D6+R772kg/zzJhW8uafP1eeB/DeRfCwzNr8flz8irmryH75Fs7VeSvij/BEyoI3tH/nk1sA/Jz/yRdn2mB/LR5xNYXg7SY2EZ+VJKrureIQ2uTco/pwODKq8byB9L8saYkRXLPcDdTcZ/AFihcD4UeCC/nlpD/v+Ah0hmh1YUyhRgvcL5m2sprIrSAK4HnskKYkLlqNP3rcD2hfMdgNubvN+78s/JJBu6Ku+3jvw6pFpvu+XzDYEPtvGzNo20St00K/WTgSub3PNgRUm30P++wGrA1vl3OxnYr13zH8hHmBRKkDc7xrPksatiG6u7G53NCT8nrfI2zHa7o2x/osE9NUMnSSHFFZn32/51HXspQD2fyGcljSCtQi+U9BTwYr250L2ooguBOyVdns/fBfwm20druaN9H3iX7ftb7P8o4DKlckHbk/yb31lD7gcl5lzhU8DFkp4g/X3XIVWwaMQkSa8iBT5MJpkjbq8n7JSg/UeF83/QpMimpNVJ7ohFG3Gtx32ARbYXSPo/0ubZaZKmNnkPM0nBME3dyGxXqj0/R3pCClokvBRKIOkB4NN03S2uq4wk3QkcSFpRVTwbGrrtSHoYeEOTfo+yfVa2H1Zj13Fyr7ipkZTJYaSVyoX1xsouXm+zvaDeXOrcN5aUvhLgVtuTGsje6pK5JfIX2Vmk97KP7bqbNkqViZ+0/XI+XwlY2/asOvIrkNz3AB50CRc+SaOAVW3fXePaLbZ3LfjjLr5EAz9cSR8jffGtT1q97kxaddf0CsifuZ+QUnC+y6k6c7PP3KWkPCHVId/H1ZDdmPTkM4ql7b371es/SITCLYGkO22/oTv3SJpaULjTbW/b4J6WlZykN9m+tVlbWbobyZbv3bBWu6vcnvIKDJJJYB1SVYi6+R0k/YmlFdWWwJMkc0Hdf3hJk4A32n4ln69I+hLYsSDzFtvXFeZUPfcuoc+StrD9QFWgR/GeKbXay5K9D3Yk2U7HSNoC+LbtmnOVtCXJe+B227/NCvJ9tr/XYIwP1Wq3fUEN2emkp7bqjHM3lnhbyyVhUijH9ZJOJuUdKCqGRv9Y/8w+oM4rp+NJXgFdKCi5mcANklpRcqeRHqubtVXG+D/SBslapJVVvdVVdyPZICnnimJcCdiYZCOs9oR4V+H1S6R0jhVq5XfojokAkk37lcUdJ9e+6vfyZuC6qjk1mgskO+yRJK+PWvfU9UtVufSGL9t+WRKShmYlXzc82ymK8LjC+aOkv3ldainWBrxs+9QS8kEmVrglqIqgquB6j3b5npHAKSRXI5FcjI6v9QhfxzxQHOjrBdldSK5KnwJ+XBBbFXh3vRV0NleUsZcuM3kF+AnbH2tjnxuRcgz/TdLKwGDbz9eR/SvJljkhn+8PHOcGEYI9ST0bveun1PwjKenQp0hK/BnSpuQ7q+R+b/t9BX/cpajXf753NMkWXu1L3GV/Qqkm3mjSZ7nVhUdAKNyOR3XCdSW9meTyczQpU1WF54E/OdURq9VfKXtpVlbvtf1sPl8duMj220u+j7oReZIuIH0JFcf4oeuHrpYKsVaqPXchyetDpGxXH7T9cA3ZNUjKcFeS0roFOKmJPf29wF9sPy/pK6Sni2/YrrlR1YqNvsFYbybZ3f9SXLXna+vaflLdiCaUdAvpff+YtMr/MMmL5cQast8hZcx7hKW/MCLSrAlhUiiJpH3oGtJYNwuTupFsWjXCdSUtFa7rJQ7nc10VSJAVQE2FS9pR/x1N7KUF1qwowiz3jFKUWl2qPCcGkRTQEw1u2abGGNs1kP8kOcQ6yz/UaE62HwF2zt4Z2H6hnixwEcmD4z35/DBSVNWeDe75qu2LlSp77ElywzoTqGfvL53eUF2rhqxHyoexGNtP5p/dCdNeyfa1kpTvH68Ult5F4ZLSbW5SrfCD5oTCLYGkM0mRR3uQwjEPBO5qclt3kk2XCdc9mORWVeSLJGf/WqxKa/bSCgslbVixL+bVU7PHomKC9QUkm27NuP/MIEmr234mj/FqGn82S4VYqypdYeW+Ol+U69r+RuH8m5KauYVV/q77AGfbvkIpBWM9ytjoK6amsaTNy1+Q3Ld+zRIvkIpctffD4ks0z0Y2Lz9NPSTpGFIwzYg6svcCryJFKrZEdxYeA5FQuOV4o+1tJN1t++uSfkhSiI3oTrLppuG6kt5B8j1dT1JxA2NVkpKrx7m1vBoayH8ZuEXSjaR/3N1Ij/ONuM/2Ugo/r7rrfQn8ELhdUuX6e4FvN+j/RpULsS6TrvAaSQcDv8/nB5ICJhrxuKSzSDmPv5cV/KAG8mU3IluqGmJ7WSqJHE9aTBwHfINkK67puUBStg8ohfMWvzAauYX19yoXbSFsuCUouHjdQYqO+i8ww/amDe4pnWxa0nHACaRosH1IkUi/tr1bQWZbYAxwEks/9j0PXF9ZLdboe4rt7Zu1VV0fSfL9hOSa1LBScTfH2JIlu/rXuUG+XpUsUqkW0hVq6Vy1w1limxwEvNBodZg37fYmRbs9JGld4PW2r2k0ZqtIusv2TpXfoZIv9e0NNtlqVdd43m1KCZrtyF1o5BYmaZrtMe0Yvz8TCrcEkr5Kcrl6K3AG6R/0nFobC4V7nif9Ay9TsmlJQ1zDLzevhAVslptqOuovg1dDJUBiE9snKfnYrmO7iymlsOp+H8nuWRxjS9s71RnjV7Y/0Kyt6vpKpHDnpjl21fPpCl8LzLY9T9I4UkTgL4t26Sr5UukNVbJqiKRZwAYkbwaRVqT/InlFHGF7ckH2J7Y/pa4+zpU5LbVqVXJnm2F7i1pj16M7C4+BSJgUSlCw7V0q6c+ktH0NNz/KPOape+G6bySFhc4i/XNtIOlD7hr2uSLJJjeEpW2s/yM9Ntfjp6TV3ltIq+nnSfbYHWvIPkHK97sf6fG9wvOkCL16LOWfm/+p65asUcp4djLpPW0saQzJk6DeI+2uwOFKicKbpitUuTBaSL+PsZI2Bc4mPT7/htrhxlAivWH+wvsdqQ5dq1VD/gpcYvvq3MdeJBv2L0h/z+Jm3q/yz5Z8nJ0S6T9YtOs3Qr1X5aJfEAq3BJKGkeyFi12GJP3MOWS0SrY7UUiVApNlbHE/AvaqrPSUKsj+liqFVfBqON/2Y5JWdpMcr5k35MfYqbmfZ9Q1aKAyxnRJ9wJvdwuO9EoFOSu22P+R/gkBXiEprnp8jeSlcEMed5pSNFU93tFsLoU51QyjpUEQA+VzF7Sc3tC2JV3p5FLXammmnW0fUejjGkk/sH1Uti8X+5+cf5aJElsdmCHpLgp5OGp94S2jXXnAEQq3HL8krdYqj3KHklYI760hWzoKyfZZ+efXu9xRnxWKj9W2/57NDPV4jaSraD2Zzvy84jQsfhxeVEe2sgLaQNKKzdyGbH8H+I6k79j+YiPZ6jnZfq7ibVDprsE4j6mrW1W9HfjjWRJGu4dyGG2z+Ug6BPggSyLVGv0NKiafJ5XcDJ+gcVXjKZJ2tN1qztknJZ1AcnGDlHzn3/nvuNTfTnWCJCrUeQr4aovzKI5TqsrFQCUUbjm2tr1l4fx6STU3d2wfmX+2nE2pytugVp9dEomQ/GrPJbkJQbK31k0UQ0pq8nZSisLKqnT3BvKnkoojriXpWyTzw1cazZPkH3qrpAksvQKql3/hqlpzaPAYP0Mp2mmwUtDDccBt9SajFt2qMqXCaDMfJpkGvuWUKGZjljyq1+KbklYjVXs4jWTjbmRyeQNwmKTHSL/PZhUcDiU9BVyWz2/NbYNJ9vUipZOW275RNSL9mtxWtsrFgCQUbjmmSNrZ9h0Akt5AY+VGlmu1ntbkGm3N+DgpEKCijG8m2enqYvufVavDum46ti9UcoB/K+kf/QA3Dwt+JB+DaM088rnC62Ekc8Fk6j/GH0tyV5tHspVeDTTye23JrSozWynV4mXAXyU9AzQMJHDJ3AUun96wVFRf9iI5ts7lh6tkSwdJqBDpR0p2vh4p0KNRqHTZKhcDkvBSKIFSxdvNST6UkNy1HiT5vdZccahOPa06q9Xqe5tGRqlQFTifDyYlkq5pn5V0Ccnuezpp5XQ8MNb2wXXkf06yS04rtI23Pb4d869z3wakx8331Lg2GPhbySeHUm5VhfvqhtFWyT1K7R3+mnmSJW1Cyq+xC+kR/3bg07ZnNhhjW5IPNKSKHtMbyG4GfJauX/KNcn7sTFptv460GTkYeLHWppa6V0z1RuAvpKeB3UlBE9Mb3TMgcQdkQe8vBynxeN2jzj33Q+MyPDXu2RqYSlpZ/YO02tuqjuwddC1nc1uDvkeSdsn/TfrQ/5q0iVNPfjbJH/iDhbYpJeb/WKP517lfNCjFQ8rZulqJ/j5Lyp07kxTtdDtwbA25wTSo1NCg/zUKx3ok17uTGsjfQcpFMCQf7ycpr3ryx5Oiu07Kxz215l+Qn0568tmJtHm6A7BDk/cwiVQhYmr+PXwY+E4d2Tvzz6n55xCaVwHp0SoX/eXo8wn0x4OU2rDV+l4t19Mq3HMbsEfhfFw9JUo3anCVnMsU0irvTyTf4yHUKJPT3fnn66eRbMWnklbet5ACPerJX076Ivp54b6adeKy8t6A5MN6Msn96W1N+m65nFGDfiY3uNZFOdG4zNHdwPDC+fBGCq7R2A3umVQ9t3p/Z1Io+ZdIpZTeRrLxf6tdn7mBfIQNtwQqUfG24Ei+CnBfdqFpNQxyuO3rC7I35MfgWrwoaXtnNzOlSgtza8zn804l2k+j9uNvPROHnHyN3yVpPKmG1WoN5l52/rC0HXwB8Fs3TqD+B5bkfqi8F9UStEu7VdVyebLt/evdUOX6N4i0Qdfof+sqSV8geRGY5EVwpXKEmO2nq4dgaTv7Quq838yfJH2CpAiLn7nqfou8lN39pkn6Pimxe73w5C+QIv3uIdlyr7B9bi1BdbPKxUAlFG45vkHyy/yb7e0k7UF6HKzFBFKC6Zur2ncjfZgbMVMpqq2y0/1+0uNwLT7FkhpcAOtSuwbX0ZJuo4VNPgAlJ/51yN4MALbHS1pIWrG2a/7YviC7auHGpXL2B9a3fUY+vwtYk/SP3ChfRRm3qqLLUyV3RE37doGi698CUhBKLVfBChVPgUpOioryPJj0Xqptv78g1Yj7Yz4/gLS6r0clB0JxM7JWv0U+QFKwx5A8JjYgha8vpur3f07ePFsT2EHSs7Yvqe7U9q75Z/jjQpgUyhyUqHgL/JkUT1/d/npSvtpG46xOekyeQrJ//gRYvUpmR1KILSQ3p2NIFQtOJ+WJre7zUyTb5SzSI+F2TebQo/PPciIV5ZwDPE0KRf0PKZKqVr+3AhsUzqeRdso3BK5tMJ8HSKvCR0iP5w0rD5M8Gk7Ov6vraWAvrXP/YOCwGu2L/2b5/EOkL7RT6/zNiu91e5InxHF5fvu2+bN9fLO2Zfj9d8s2PhCPPp9AfzpImY5GZKX2W9JOcz3b6sQG/XQp6Z3bh2XFeDqpMu0KDfqYUvknJe36PkEK3/wGKayz3n0bkVaDU7MiOpHkT9mr88/ynyE95m9caNuE5Ob16WZzIrkXVV7fUUN+w8J7brjJScpF8bX8O7mF5Fb1WJP5r0pKhXk6yZYp0hffo8Dly/o3y3MZVaP9I8AjNdo/X3j93qpr327yXrpshFJlwy37+6+SbYttvL8ffT6B/nSQ0tcNyt/YH8r/lF1WJln2oQb9PFyn/Xckr4GjSH6gP2nQx/TC6zOA8YXzaS2+n+2y4l3Y2/PP8lOBkTXa16z+Z280br5WSwFNKby+tMlcFgE3ApsW2mY2uedy4Pz8fn9PCjW+ERjTjr8ZKRfD3yl8IZLsp/eQHu0bvd8p9a5VtR9C2hB9hrTarhzXU7VqLfv7r7p+EylK89riOK18TgfSETbcFqhh8IcldrcTJT0CfNn2tYXrkyQdYfucqr4+Rv0Ahy2d/RKz/2uj5OaDtSSD2FtZOkdt3b+rUrLud5DshW8lKYnxNUR7ev6QVsBdUj3a/k+d8OQ768zpqDpjFTeWGtkvIdkrDyZFD/6FtKHVaGMKUga1yvs9l2Sb39A1cmtkSv3NbF+plOzlKkkHAB8juXrt7trpN1Xnda3zCrfleY9kaVv08yTzS5Gyv/8ipcOBByKhcFvADQz+2RF/a5JvazHn6qeAPypVbagoqLEkp/J31+lucVpFp2Qojab1W1LSkzkkr4Sb83w2pUb5FqVE3YeQVk13kRTKkbZfrJbtpflDSlJT5tqngctyWG8l+c8OwFDSRlI1rvO6q6B9We57OLA/6f2vJelnwB9dO7dt8f0ulDS7gbKFkn+z3O+1kj5M+mK8DXhLgzEavd+a798p0uwxUhBGM8r+/ovjlEmOM2CJSLM2Ieko5+QzVe17sEQRz7B9XYM+FrLEFUmkEuMvUceFJkcHrQtcU1GcOcpohKuykUm6jhQGe2md1VG9OfXk/IvyS10ipb6smQBG0ltY4opXd06F/otzqfTfZT417l+d5G1wkGsUqCz7fvM9Zf5mxdSGQ0kKvuIS1uj3Wev91vx91nl6q9xT7z209Pvvbv8DmVC4QRAEvUSjuktBEARBGwmFuwxIalZMcZnke2OMkA/53h6jO3MaKITCXTbKfnC680Hr6TFCPuR7e4xQuEEQBEHPEptmLTB4leEessbqXdoXvvAig0d0zcky9LHapcLmM48VGFrzWj3K3tM2+ZWHdW0D5i94iRWGrNz1wku1PZXaNR+tWLtizSsL57Li4JW6XqjjklZXvg715D2vtkdbvfkvWLN27p4Fc19kyEpdrw2ZU+cz5JdZQbX/NtT4X+5Pn7nneWaO7TVb7qgGb99juP/7dN18+ouZfPe8q23vvSxjdYfww22BIWuszjpfq5dAvyubfbSl/DAdjbbokgCtIZ46o4dmkhiyzvql5D20UUmxZWfhw4+Wkn/qoDeWkl/73PLFPzxvXnOhDuZvvqR09Ylq/vv0Qu66esOmcoPXfWjkso7VHULhBkEwYDCwqH6N0z6nRxWupBds16uOGgRB0FaMme/mJoW+Ila4QRAMKDp5hdvjXgqSRki6VtIUSffkJMZIGiXpAUkXSrpf0iVK5ZaRdKKkiZLulXS2clC+pBskfU/SXZL+Lmm33D5Y0sn5nrtzMg0krSvpJknTcl8V+b0k3Z7ndLFyscMgCPo3xix086Ov6A23sJeBd9venlQS+ocVBUqqgPtT268D/gd8IrefbntH21uTYsL3LfQ3xPZOpOQiX8ttHwWes70jKcnzEZI2Bg4FrrY9BtiWVD5kJPAVYM88p0mkvKxLIelISZMkTVr4Qr38LkEQdBqLcNOjFSTtLelBSQ8rlUSqvv7jvJiblheAzzbrszdMCgK+LWl3Us7R9UilZwD+6SW1q35Nymb/A2APSZ8n5Z99NTCDlLMTltSymkwqAw2wF7CNpAPz+WrAaGAicF5O9XeZ7WlKpa+3BG7Nen9FUiWEpbB9NnA2wNBR64fvXBD0AwwsbFGhNiJnATyDlFh+NjBR0gTb9y0ey/50Qf5YUn7phvSGwj2MXPfI9nxJs0iVAaBGCjlJw4CfAmNt/1OpcGHR8bDi+7KQJfMXqQzK1dWDZ0W/D3C+pB+REi3/1fYhy/zOgiDoOFpdwTZhJ1LC9ZkAki4ipe28r478ISx54q5Lb5gUVgOeysp2D1J5kwobSqrk4TyUVNqkolznZNvqgTTnauDjeSWLpM0kDZe0EfDvnDD5XFJdqDuAN+UcpGS5zZbxPQZB0AEYmG83PYCRFZNhPqrDjdcD/lk4n53bupD1zMakmoIN6bEVrlJlgXmkxNx/knQPyV76QEHsQeCTks4jfXP8zPZLks4B7gX+RTILNONcknlhSrYP/4eUEHkc8DlJ84EXgA/magKHA7+VVAl3+QqplEkQBP0Y41ZNCnNsj23TsAeTatI19UfrSZPCVqQ6R3OokU1e0ihgge0uZcZtf4WkBKvbxxVezyHbcG0vAr6UjyIX5KO6n+tIm2tBEAwkDAvbs+PyOKlUfIX1c1stDgY+2UqnPaJwJR1N2gD7VE/039sMfeylARGuW4ZBL8wtJd/TruYL/jm7lPyGd9bOXVCPJw6oW0WpLQx7ppxvqF732tJjeFo98+LyQ4o0awsTgdHZ2+lxklI9tFpI0hbA6tTYeK9Fjyhc22cCZzaRmcXSNcCCIAiWEbGwae3P5uSafMeQ9ocGA+fZniHpJGCS7QlZ9GDgIreYBSwizYIgGDCkTbNlV7iQqiYDV1a1nVh1Pr5Mn72eD1fSOpIukvSIpMmSrsxBBn+uI3+upC17e55BEPQ/kh+umh59Ra+ucLMHwR+BC2wfnNu2Bfard4/tj/XS9IIgGAAsatMKtyfo7RXuHsD8bOMFwPZ04GZgRM6nUMmvUMyfMDa/fkHStyRNl3SHpLVz+5qSLs25FCZKelNuf3Mh9G6qpFVy++cKeRe+3su/gyAIeohOX+H2tsLdmhSSW4vtSF4NWwKbAG+qITMcuMP2tsBNwBG5/RTgxzmXwntIfrkAnwU+mXMp7AbMlbQXKex3J2AMsEOORluKYi6F+fTvxM5BsLxgxEIGNT36ik7aNLvL9mwASdNIPra3VMm8AlRsvZNJcc4AewJbLsmJw6o5Su1W4EeSLgT+YHt2Vrh7AVOz7AiSAr6pOFAxl8KqenXkUgiCfkInmxR6W+HOoH6obnEZWcyTUGR+wf2iKDMI2Nl2dWGt70q6AngnKVnN20l5F75j+6zuvIEgCDoXI17x4L6eRl16e219HTC0GLcsaRvS4/6ycA2wuOiYpDH552tt32P7eyRH5i1IfnUfqeTAlbSepLWWcfwgCDqAFPgwqOnRV/TqCte2Jb0b+ImkE0i5cmcBly1j18cBZ0i6m/SebgKOBj6VE+YsIq2ur7I9T9LrgNuzCeIF4P3AU8s4hyAIOoC+3BRrRq/bcG0/AbyvxqVzCjLHFF6PK7weUXh9CXBJfj0HOKjGWDVL7do+hbTRFgTBAMIWC913K9hmdNKmWceiFYYwZOTazQUzC/7173L9Dx3aXKhA2XLYQ9Zpfe6Leb5/V7n4xxvKzX/eO8rF1gx98l+l5P+7TblV16vuLZ8RYPCrVisl/9Iu5bKSrnx7uYR6C599rpR8u1gUK9wgCIKeJ22ada5a69yZBUEQlKSyadaphMINgmBAsbCD/XA79qtA0kItKW9+sXIJ9Tqyh0s6Pb8+WtIHG8iOl/TZnphzEAR9S6dHmnWswgXm2h6TS6W/QnLzaortM23/smenFgRBp7LIg5oefUUnK9wiNwObSnq1pMty0pk7ctDEUhRXsJKOk3Rflr+oILZlToozU9JxvfUmgiDoWVLyms5d4Xa8DTcXo3wH8Bfg68BU2wdIegvwS1ICmnp8Adg4Bzu8qtC+BSlz2SrAg5J+Znt+1bhHAkcCDBs8giAIOh8j5kdob7dYKSexmQT8A/g5sCvwK1hcCHINSas26ONu4EJJ7wcWFNqvsD0vB0w8BXRxVLV9tu2xtseuOGiltryhIAh6FhsWelDTo6/o5BXu3JxWcTGFbGCtsg+wO/Au4MuSXp/bW0mUEwRBv0MdHfjQySvcWtwMHAYgaRyptvz/aglKGgRsYPt64ARgNVIqxiAIBigmVrjtZDxwXk5S8xLwoQayg4FfS1qNlJLxVNvPdmOVHARBP6IvN8Wa0bEKt5ioptD2NHBAjfbzgfPz6/GFS7vWkB1fdd68VLuNFyxoKlZBO2zVsiyAJ88oJV+WMnNfXhl+9+Plbth4o3LiX7i9lPysE99YSh5g44vmNxcqMPSqiaXk/33ULqXkR55V7j23A6O2JSCXtDcpydVg4Fzb360h8z7SQtDAdNuHNuqzYxVuEARBWVKZ9GVXa5IGA2eQqsrMBiZKmmD7voLMaOCLwJtsP9NKXu3OXXsHQRCUpnkByRbz5e4EPGx7pu1XgIuA/atkjgDOsP0MgO2mObVD4QZBMGAwLUeajawUic3HkVVdrQf8s3A+O7cV2QzYTNKtORBr72bz63WTgqQvA4eS3LEWAUfZvrPE/WOA19i+Mp+PA16xfVs+Pxp4qV54r6TxwAu2f9D9dxEEQafS4gp2ju2xyzjUEFIB2nHA+sBNkl5v+9lGN/QaknYB9gW2z9FfI4EVS3YzBhgLXJnPx5HK5NwGKZdCWyYbBEG/w1a7ciU8DmxQOF8/txWZDdyZo1QflfR3kgKuuxvZ2yaFdUnfLPMglcax/YSkHSXdJmm6pLskrSJpmKRfSLpH0lRJe0haETgJOChnEjuBlNTm0/l8t8ilEATLL2nTbHDTowUmAqMlbZz1zsHAhCqZy0gLPvLicTNgZqNOe9ukcA1wYv4m+BvwO+D2/PMg2xNzqO5c4HhS3cnXS9oi37sZcCIwtlL3TNJKFEwEkt5aGK89uRQGRbxEEPQP2lPTzPYCSceQqnwPBs6zPUPSScAk2xPytb0k3UcykX7O9n8b9dvbVXtfkLQDqSz6HiRF+y3gSdsTs8z/ACTtCpyW2x6Q9BhJ4ZahkkvhMpauDHxFXmXPk1TJpTC7aq5nA2cDrLbCmi45bhAEfUDaNGuPH27eJ7qyqu3EwmsDn8lHS/RF1d6FwA3ADZLuAT7Zg8NFLoUgWM7o5EizXp2ZpM2zs3CFMcD9wLqSdswyq+SUjMW8CZsBGwIPAs+TTAEVqs8rY0UuhSBYzqhEmjU7+oreXtmNAE7L9tQFwMMkO+kvcvtKJPvtnsBPgZ/lVfAC4PBsi70e+EJO3fgd4E/AJZL2B44tjBW5FIJgOSSKSGZsTwZqBYnPAXau0f7hGn08DexY1Vys/HBz4XV7cikMHoxW7bKIrsugmU+0LAvJptGT+IUXS98z59DtSsm/+rzej5tvxBOfK5eLYP3TppSSH/SadUrJD96y3PbDmtPL579Y+PdHSt9Thr7IjVAWG+YvCoUbBEHQ4ySTQijcIAiCXqHFSLM+IRRuEAQDhna6hfUEDdfekq6X9Paqtk9JelTSF5rcO05S+aSeQRAE3Ub9ukz6b0khbUUOBj5UKxlvFeOovUEWBEHQYyzKdc0aHX1FM4V7CbBPjiVG0ijgNcBrJZ2e29aUdKmkifl4U5arznFwvqRTc86EmZIOzPePkHStpCk5b8L+lbEkPZDv+7ukCyXtmVOhPSRppyw3XNJ5OQfD1ML9W+W2aTmXwujc/v5C+1lKiYaDIBgAJC+FwU2PvqKhws0uWHcB78hNBwO/J5lKKpwC/Nj2jsB7SKUoZgFn5vYxtiuuWuuSXLX2BSor5JeBd9venhTu+0MtcZbdFPghKffBFqS0jrsCnwW+lGW+DFxne6d8/8mShpMU/im58u9YYLak1wEHkTK0jyF5ZB1W671LOlI5V+YrC19q9GsKgqBDGAiBDxWzwuX550eB1xeu70nKvlU5X1VSvYiuy2wvAu6TtHZuE/BtSbuT8uOuR8ptAPCo7XsAJM0ArrXtHAwxKsvsBexXyRAGDCNFpd1OCuddH/iD7YdyYpsdSOUyAFYCamZpXyqXwrB1IpdCEPQTOrlMeisK93Lgx5K2B1a2PbmQkwDSKnln2y8Xb6oT0VXMYVAROAxYE9jB9nxJs0hKs1p+UeF8UWHuAt5j+8Gqse6XdCcpn8KVko7KshfY/mKjNxwEQf+kX3spQMrwBVwPnEda7VZzDYWQWqWKDFAnx0ENVgOeysp2D6BcOdSUIu3YihlC0nb55ybATNunkr40tgGuBQ5ULvYm6dWSyo4XBEEH05+9FCr8FtiW2gr3OGBs3pi6j2Q7hZTj4N2VTbMGfV+Y778H+CDwQItzqvANYAXg7mx2+EZufx9wb865sDXwy1xx8yvANZLuBv5KsisHQTAAsMUCD2p69BUtBT7YvowlJgBsnw+cn1/PIW1EVd/zd+rnOMD2iML99Qreb12QP7zwelblmu25wFE1xv8uSzbmiu2/I+XhDYJgANLJJoWINGuB+auswL/f0vpCeI1zZ5Xqf9Dw4aXkF71YLhnNom1GNxeqYuTv7i43RukRepb1fzq9lPzz+2xbSn74pS3XPe0Wi173htL3lP0caUQ5+YUbrd1cqMhd95STbwOdbsMNhRsEwYAiFG4QBEEvUPHD7VQ6No+ZpHUkXSTpEUmTJV2ZKz8sa7+Lq/oGQTDw6OTQ3o5c4WYXrz+SfGYPzm3bkgIi/t6XcwuCoHOxYUEHJyDv1JntAcy3fWalwfZ04G3ZzWyapMcl/QLq50eQtHfO0TBd0rWF/reUdEPO6XBcr76zIAh6lHaF9mb98aCkh2tlR5R0uKT/FHTSx5r12ZErXJLL1+Tqxlyi+ESlmmg3A6dX5UeYL+mnwGGSrgLOAXa3/aikVxe62oKk1FcBHpT0M9vzi2NJOpJUb40VRqze9jcYBEH7aZcNNy/azgDeBswmpQOYkH35i/zO9jGt9tupCrcu2dzwa+BHOcz4GGrnR9gZuMn2o7A4EU+FK2zPA+ZJeopkqphdHKeYS2HlNTeIXApB0E9wezbNdgIetj0TQNJFwP5AtcItRaeaFGaQlGgtxgOzbf8in1fyI4zJx+bVhSJrUMzRsJB++MUTBEFtWtw0G1nJBpiPI6u6WQ/4Z+F8dm6r5j05yvYSSRs0m1unKtzrgKHFX4KkbSR9lZSdrGh3rZcf4Q5gd0kbV9p7bfZBEPQJdss23Dm2xxaOs7sx3J+AUba3IaUJuKDZDR25ssspGN8N/ETSCaScubOAlUnfMndl88EE2ydKquRHGATMBz5p+46ssP+Q258i2WOCIBiwiIXt8VJ4HCiuWNfPbYux/d/C6bnA95t12pEKF8D2E6QENK3I1syPYPsq4KqqtvFV51sTBMGAoU023InA6PyE/DgpF/ihRQFJ69p+Mp/uB9zfrNOOVbidxKCFMOyZHswWsHBhz/UN3Ypp/9flW5SSX2v/skneepay+SZeeE25siurDBvWXKjAopdfbi5UoDu5Gkp/Qkv+jgYvWFBKvoc/1TVpVy4F2wvyhvzVwGDgPNszJJ0ETLI9AThO0n7AAuBp4PBm/YbCDYJg4OBkx21LV/aVwJVVbScWXn8RKFXMIBRuEAQDiv5eYmcxkhYC9+T77ieVS2+pwmKuBPGa/K0RBEHQdty+TbMeoezM5mZf162BV1hS3aEhkoYAY4B3lhwvCIKgFHbzo69Ylq+Cm4FNs9/rZdn59w5J28DirFy/knQr8CvgJOCgHHN8UHXWLkn3ShqVX381xzDfIum3Fbmc/2Bsfj0yF5xE0mBJJ0uamOdxVG5fV9JNecx7K6V+JO0l6facZ+Fi1a8yHARBP8NW06Ov6JbCzSvWd5DMC18Hpmbn3y8BvyyIbgnsafsQ4ERS3PGY7MZVr+8dgfeQaqi9AxjbwpQ+Cjxne0dgR+CI7M5xKHC17TG5v2mSRpLqmu1pe3tgEvCZGvM4shKFMn/eCy1MIQiCviatYDtX4ZbdNFtJqSgjpBXuz4E7SQoS29dJWkPSqllmQq45VoY3AZfnsusvS/pTC/fsBWwj6cB8vhowmuRLd56kFYDLbE+T9GbSF8GtOXhiReD26g6LuRRGvDpyKQRBf6GTE5CXVbhz82pxMVlp1aORo98Cll5ht+LYWLynKC/gWNtXV98gaXdgH+B8ST8CngH+mlfdQRAMMPrSRtuMdmzn3QwcBiBpHClG+X815J4npUOsMAvYPt+3PbBxbr8VeJekYdm2um/VPZWkNgcW2q8GPp5XskjaTNLwnFPh37bPIYXebU/KsfAmSZtm2eFqQyWJIAj6HiMWLRrU9Ogr2jHyeGAHSXeTypJ/qI7c9aTE39MkHQRcCrxa0gzgGHIlB9sTgQnA3aSw3HuA53IfPyAp1qnAyELf55LSpk2RdC9wFmn1Pg6YnuUPAk6x/R9SRMhv85xvJ+XHDYJgAOAWjr6ilEnBdpfd/Jxn9oAa7eNryO1YJbZXnaF+YHu8pJWBm8jJyG0/AGxTkPtKbl9E2rD7UlU/F1Ajg4/t62rMJQiC/o7blkuhR+jUSLOzJW1JstNeYHtKX05m0AIz7On5zQUr8j0cZ1+WsvMBWOegWaXkezDTRK8wb42+nsHS/OfoXUrfs9adtSx59fHUGaXkF224bil5/vt0c5meoINtuB2pcG0f2lwqCIKgK7HCDYIg6AUMLFrUuQq3I4OOJS3Mm2szlCru/r+cRLzZfdU23Foy5xf8dYMgGEgYsJoffURHKlyW5GzYilSl4R3A11q4r6nCDYJgYDNQcyn0CrafIpUrP0aJwyWdXrku6c+Sxkn6LjkSTtKF+doHc26F6ZJ+Veh2d0m3SZoZq90gGGB0sF9Yv7Dh2p6pVCd+rQYyX5B0TCUSTtJWJLexN9qeU1VEcl1gV5L/7QTgkur+cj20IwGGDn1Vm95JEAQ9S9/mSmhGx69wl4G3ABfbngOL/YArXGZ7ke37gLVr3Wz77EpFzxVXHN4L0w2CoC3ECnfZkLQJqUTSU3QvB0M184rdL8PUgiDoJAwOL4XuI2lN4EzgdNsm5VMYI2mQpA2AnQri8yv5FIDrgPdKWiP3UzQpBEEwYFELR9/QqQq3svk1A/gbcA0p7y6k5DaPknInnAoUo9DOBu6WdKHtGcC3gBslTQd+1GuzD4Kg72iTSUHS3rkQwsOSvtBA7j2SXCmO0IiONCnYrluzOq9yD6tz7QTghMJ5l1wKtg+vOo9qD0EwkGiDjTZv0p9BckudDUyUNCHv+xTlVgGOJ+UFb0pHKtyOY5EZ9PLC1sV7ODdCWTptPt1h0Morl7ths1GlxO8/8qel5N8+fkwp+bLzX/PMLjnxm9LTe0Flcy/0CZXAh2VnJ+Bh2zMBJF0E7E96si7yDeB7wOda6bRTTQpBEATdok2BD+sB/yycz85ti8l5vDewfUWrc4sVbhAEA4vWvBRGSppUOD87l9VqiZxq4Eek3Not07EKV9LawI+BnUllcV4Bvm/7j8vY7zjgs7b3bSIaBEE/RK2tYOfYbrTJ9TiwQeF8/dxWYRVga+CGXGZsHWCCpP1sFxX5UnSkSUHpHVwG3GR7E9s7AAeT3nRRrmO/MIIg6ANa8VBoTSFPBEZL2ljSiiT9M2HxMPZztkfaHmV7FKl0V0NlCx2qcElRYq/YPrPSYPsx26flXAoTJF0HXJtrkp0n6S5JUyXtD2mXUdLJkibmfApHVQ8iacd8z2t7760FQdBztJAprIVNNdsLSKW/rgbuB35ve4akkyTt193ZdeoKcSuW9q+tZntgG9tPS/o2cJ3tj0h6FXCXpL+RXMees72jpKGksujXVDqQ9EbgNGB/2/+oHqCYS2HYiqu1630FQdDTtMldw/aVwJVVbSfWkR3XSp+dqnCXQtIZpGQzr5B84/5ayI2wF7CfpM/m82HAhrl9m0I2sNWA0bmP15GCJPay/UStMbMB/WyAVUes18FFO4IgWIoOrvfUqQp3BvCeyontT0oaCVTsIy8WZAW8x/aDxQ6yHfhY21dXtY8DniQp5u2Amgo3CIJ+SPv8cHuETrXhXgcMk/TxQls9z/GrgWOzgkXSdoX2j1dyK0jaTFIl7dezwD7Ad7ICDoJggCA3P/qKjlS4OXz3AODNkh6VdBcpRPeEGuLfAFYg5VCYkc8BziVFhUyRdC9wFoUVve1/A/sCZ0h6Q0+9lyAIeplIz1ge20+SXDFqcX5Bbi7QxQPB9iJSyZ3qsjs35IO8WbbVMk82CIKgBTpW4XYSi1YYxNx1W0+7u8qrynk1LJpbLteB581rLlRAQ4eWkgcYvG7NvOx1WTCri6NHW1n00kul5Ic8+0Ip+e2++YlS8q8ZNbuUfOnfz06vLycPDPnHU6XkF/zr3+UGKDunu+4pJ98m+tJk0IxQuEEQDBxMq6G9fUIo3CAIBhYdvMLtmE0zSQtz0vHKMaqHxhkn6c890XcQBH1PJ3spdNIKd26l4m412eVLeSMsCIKgPrHCLY+kUbm8xS+Be4ENJH2ukBvh6wW5+yWdI2mGpGskrZSvbSrpb5KmS5pSyJkwQtIlkh6QdGHFhzcIggFAB7uFdZLCXalgTqikYBwN/NT2VsDm+XwnYAywg6TdC3JnZLlnWRKldmFu3xZ4IynCDFKE2aeALYFNgDdVT0bSkZImSZo0f165He8gCPqGVswJYVJILGVSyDbcx2zfkZv2ysfUfD6CpGj/ATxqe1punwyMyrWG1qvkz7X9cu4X4C7bs/P5NGAUcEtxMsVcCiNW36CDH1KCIFiK8FLoNtU5E75j+6yiQFbMRcfUhcBKTfqtlu/030MQBC3SyX64nWRSaMbVwEckjQCQtJ6kteoJ234emC3pgCw/VFLJSoRBEPQ7OtiG229WdravkfQ64PZsFngBeD9phVqPDwBnSToJmA+8t8cnGgRB39HHNtpmdIzCtT2i6nwWqWZQse0U4JQat29dkPlB4fVDpOoRRWaScylkmWO6O+cgCDqQULj9m8HzFjLi4edall/4bOuyncp/3/SaUvKr9XAuhbKUzV0w8u5Xl5J/fsw6peRXKSUNr6w4uOQd3ciNUJJ5a7SeTwSgfAaP9qAO9tbvTzbcIAiCfk2scIMgGFh0sEmhI1e4kg6QZElbdOPemlEKudrmnss+uyAIOpYOD3zoSIULHEIKRDik+oKkbq3KbZ9o+2/LOrEgCDqcNrmFSdo7pxd4WNIXalw/WtI9OTr2FklbNuuz4xRu9rPdFfgoueJDzvB1s6QJpLI5SLpM0uScP+HIqj5+nNuvlbRmbju/UsFX0o6Sbss5Fu7KUWlBEAwE2qBwJQ0mVQh/BykFwCE1FOpvbL8+R8h+H/hRs347TuEC+wN/sf134L+Sdsjt2wPH294sn3/E9g7AWOA4SWvk9uHApJxX4Ubga8XOJa0I/C73tS2wJzC3ehLFXAqvLHix+nIQBB2ISF4KzY4W2Al42PZM268AF5F002Js/69wOpwWVHknKtxDSG+O/LNiVrjL9qMFueMkTQfuADYg5VWAVJX+d/n1r0mr5SKbA0/angjpl2Z7QfUkbJ9te6ztsSsOGV59OQiCTqR1G+7IyoIqH0dW9bQe8M/C+ezcthSSPinpEdIK97hm0+soLwVJryYFKrxekoHBpG+NKyjkVcilzfcEdrH9kqQbgHpOgh28ZxkEQdtp7T9+ju2xyzyUfQap8vehwFeADzWS77QV7oHAr2xvZHuU7Q2AR4HdquRWA57JynYLYOfCtUG5H4BDqcoCBjwIrCtpRwBJq3R3Iy4Igg6kPZtmj5OenCusn9vqcRFwQLNOO03hHgL8sartUrp6K/wFGCLpfuC7JLNChReBnSTdS1otn1S8MdtjDgJOyyaJv1J/dRwEQT+jTW5hE4HRkjbO+z4HAxOWGkcaXTjdB3ioWacdtbKzvUeNtlOBU6va5pF2D2v1MaJO++GF1xNZelUcBMFAoQ1GRNsLJB1DylI4GDjP9oycCGuS7QnAMdm3fz7wDE3MCdBhCrdjmT8fPfGfvp5Ft1mwy1al71njtiebCxXHKD1Cz/LQL7cvJb/pmY2SznVl5SumlZIv1zt41MiSd/Q8K93yQCn5Pklp4PblUrB9JXBlVduJhdfHl+0zFG4QBAOLDt4mD4UbBMGAopPz4fbJppmkhTkc7l5Jf5L0qjb2fW4rIXZBEAxQOrjiQ195Kcy1Pcb21sDTwCfb1bHtj9m+r139BUHQj2hF2S6HCrfI7eQIDkk3SBqbX4+UNCu/3irnPJgm6W5JoyUNl3RFzodwr6SDavTxsxxFMkPS1ysDSpol6euSpuTkE6WzkgVB0HmIzs4W1qc23Jwg4q3Az5uIHg2cYvvC7BM3GHgn8ITtfXJfq9W478u2n87jXCtpG9t352tzbG8v6RPAZ4GPVc3tSOBIgGGDanqaBUHQgYQNtysrSZoG/AtYmxR80IjbgS9JOgHYyPZc4B7gbZK+J2k327Xq2rxP0hRgKrAVKetPhT/kn5OBUdU3LpVLYVDERQRBvyFMCl2Ym1OabUR6CqjYcBcU5rRYy9n+DbAfKavXlZLekrOJbU9SvN+UtNg/DkDSxqSV61ttb0PKx1DUnPPyz4WEt0YQDBxC4dbG9kukDDv/L+czmAVU0jFW8iEgaRNgZo46uxzYRtJrgJds/xo4maR8i6xKCvN9TtLa1IlMC4JgANHhFR/6fGVne6qku0n5En4A/D7bT68oiL0P+ICk+SQzxLeBHYGTJS0ihdZ9vKrf6ZKmAg+Q0qzd2uNvJgiCvqeDbbh9onCr8x3YflfhdJvC66/k698lJakpcnU+qvseV3h9eJ3xRxVeTwLG1ZILgqD/0cll0vt8hdsfmLfWMB49avOW5Tccf1up/he8dYfmQgWGXDu5lPzgG6aUkgeY9cU3lpJf/zuPlR6jJ9niM+XmM2/bUaXkNf+VUvJl6c7fbNDKK5eSX/TSS+UGWFg2I0Tf0MleCqFwgyAYOPTxplgzQuEGQTCw6GCF2+teCpK+nCO/7s6RY29oc/8Nn+clvdDO8YIg6Bwi0qyApF2AfYHtbc+TNBJYsZ1j2C5nfAyCYEChRZ27xO3tFe66pJDaeQC259h+Iuc2+H7Oa3CXpE0BJL1L0p2Spkr6W/anRdJ4SeflvAkzJS2ulllZwUpaV9JNhaxkuxVkvpVzMNxR6TMIggFAJK9ZimuADST9XdJPJb25cO05268HTgd+kttuAXa2vR2pSNvnC/JbAG8n1Y//mqQVqsY6FLg6R7RtC0zL7cOBO2xvC9wEHFFropKOrJRQXvjii7VEgiDoQMKkkLH9gqQdSFV49wB+J+kL+fJvCz9/nF+vn2XWJZkeHi10d0VeKc+T9BQpJ8PswvWJwHlZEV9me1pufwX4c349GXhbnbmeDZwNMGy9DTr3GSUIgqXp4P/WXt80s73Q9g22vwYcA7yncqkoln+eBpyeV75HUTsXAtTIh2D7JmB3Umnj8yV9MF+ab9v17guCoH/TySvcXlW4kjavKi08Bqh4qB9U+Hl7fr0aS2rBN62IWTXWRsC/bZ8DnEvXXAtBEAxEwoa7mBHABZLuy/kTtgTG52ur57bjgU/ntvHAxZImA3NKjjUOqORTOAg4ZdmmHgRBx+MU2tvsaAVJe0t6UNLDBdNn8fpnKrpM0rV5kdeQ3rbhTga6uG1JAjjZ9glV8peTsoNV9zO+6nzrwusR+ecFwAU17h1ReH0JcEnJtxEEQYdS8cNd5n5S0YIzSHs8s4GJkiZUle+aCoy1/ZKkjwPfZ8mTek3CftkCQ+e8wiY//0fL8gtK9l82N0JvsOFVz5aS7+l8IYM337SU/Kz3rVVKfqM/PVtKXiPXKCXvl+aWkn/2gG2aC1Wx6m/uKCU/ZIP1S8kv+Ofs5kKdgNtiM9gJeNj2TABJFwH7A4sVru3rC/J3AO9v1mlHKNxi9q4gCIJlocUV7khJkwrnZ2fPpArrkdK6VpgNNIqK/ShwVbNBO0LhBkEQtIXWN8Xm2B7bjiElvR8YC7y5mWwnVO1dKr+BpHfmwIimBuggCIJq2rRp9jiwQeF8fZZ4TC0ZS9oT+DKwXyWCthEdtcKV9FbgVODttpsmNFXabZPtDk45HARBb9KmBOQTgdG5NuLjwMGk6NUl40jbAWcBe9t+qpVOO2KFCyBpd+AcYF/bj+S2z+Q8CPdK+lRuG5VdNX4J3EsKFf6cpInZPePrhT4vkzQ5Zyc7stD+QuRTCIIBiEmbZs2OZt3YC0iBWVcD9wO/tz1D0kmS9stiJ5NcXS/OOVsmNOu3U1a4Q4HLgHG2HwDIIcAfJhmqBdwp6UbgGWA08CHbd0jaK5/vlOUmSNo9R5p9xPbTklYiuXVcavu/LMmn8GVJ3yflU/hmcUJZQR8JMGzwKj389oMgaBftiiSzfSVwZVXbiYXXe5bts1NWuPOB20g7fRV2Bf5o+0XbLwB/IOVgAHjMdsUHZq98TAWmkJLaVKLZjpM0neSysUGhvTqfwqjqCdk+2/ZY22NXHLzSsr/DIAh6hw6ONOuUFe4iUmXeayV9yfa3m8gX03cJ+I7ts4oCksYBewK7ZMfkG1iSiyHyKQTBAKRdgQ89RaescLH9ErAPcJikjwI3AwdIWlnScODdua2aq4GPSBoBIGk9SWuR8jA8k5XtFsDOvfJGgiDoO2y0qPnRV3TUyi7bW/cm5ak9HjgfuCtfPtf2VEmjqu65RtLrgNtziPALpIiPvwBHS7ofeJBkVgiCYKDTwSvcjlC4VfkN/glsXLj8oyrZWcDWVW2nUDs5zTtaGC/yKQTBAKKTTQodoXCDIAjagoEOrmkWCrcVDCxq3Zt68Oqrl+p+4TPPlJxQOYZsMqr8Tf97qZR4T0eeLHzw4VLyG51cLtHKY58tly55w6tUSt6TZ5SS18JS4gAMGjasuVCBsslotMNWpeTLvue20bn6NhRuEAQDi042KfSpl4KkhTlCo3J0SfJboq9Ktd7XSKprk82Ravd2d5wgCDqb8FKoz9xcVbdt2H4COLCdfQZB0E/o48CGZnSMH24RSbMkfV3SFEn3ZD9aJK0p6a85N8K5kh6TNLLq3sUrWElbSborr57vLtRTGyzpnNzPNTn0NwiCfk7OZtX06Cv6WuGuVGVSKJanmGN7e+BnwGdz29eA62xvRXLl2rBJ/0cDp+RV9FiWlFEfDZyR+3mWJZWDgyDo7yxq4egjOtmk8If8czLwf/n1rqSIM2z/RVKz7f3bgS9LWh/4g+2HcnDEo7anFfofVX1jJK8Jgv5JX65gm9HXK9xGVJL5djvXge3fAPsBc4ErJb2lqu+6/S+VvGZQWByCoF/QSuKa5ahM+rJyKynJDTktY0OHV0mbADNtn0qq/lu+Ml8QBP2Izs6l0NcKt9qG+90m8l8H9sqbYu8F/gU830D+fcC9kqaRwoF/2Y5JB0HQwbQhAXlP0ac2XNuD67SPKryeBIzLp8+Ryu8skLQLsGOljlAlP0Ix14Lt7wLVSvxpCrkYbP+gDW8lCIJOwG0rsdMj9PWmWVk2BH4vaRApifgRfTyfIAg6jQ7eNOtXCtf2Q8B2vT3uopVW4KWtX9Oy/IpXT2ou1IssmDmr9D0vHviGUvLDuzFGGV55e7mK1ivd9Ugp+VFn3F9Kvmz+i4cuKJerYfSHymcT7emFXZ/lRihL5+rb/qVwgyAImqESiaZ6m1C4QRAMHEyfBjY0o6+9FLpQSEIzStKhLcgXQ3nHSjq1p+cYBEFnIpqH9bYaGCFpb0kPSnq4VmItSbvn9AMLJLWUv6XjFG6BUUBThVvE9iTbx/XMdIIg6Be0wS1M0mDgDFLVmC2BQyRtWSX2D+Bw4DetTq2TFe53gd2yf+6n80r25vyNMkXSG6tvkDRO0p/z650k3S5pqqTbJG2e2w+X9AdJf5H0kKTv9/L7CoKgJ2mPH+5OwMO2Z9p+BbgI2H/pYTzL9t2UMGJ0sg33C8Bnbe8LIGll4G22X85Zv35LSkhTjweA3bLP7p7At1mSpGYMydthHvCgpNNyLbXFFHMpDB32qra9qSAIepDWbbgjJRXdic62fXbhfD2gqBNmA+Vcd2rQyQq3mhWA0yWNIeU/2KyJ/GrABVk5O99f4VrbzwFIug/YiKV/ueRf/tkAq6y2fgc7mgRBUKRFL4U5tsv5GraB/qRwPw38G9iWZAp5uYn8N4Drbb87l1a/oXCtafKaIAj6I20L3X0c2KBwvn5uWyY62Yb7PFDMi7ga8KTtRcAHgJphwVXylV/Q4W2fXRAEnYdplw13IjBa0saSVgQOBiYs6/Q6WeHeDSyUNF3Sp4GfAh+SNB3YAnixyf3fB74jaSqxgg2C5Yc2JCC3vQA4BrgauB/4ve0Zkk6StB+ApB0lzSYl0jpLUtNQvI5TRIUkNPOBt1RdLqZXPCHLzWJJspobyKYD27eztJ33K7n9fOD8wnj7tm3yQRD0Oe1KQG77SuDKqrYTC68nkkwNLdNxCrcTGfTivFKx+QtL9j9k441KyS949LGSI5Rn1RlPl5Iv+54HDRtWSr5sfgqX7P/Zd48pJb/6Xx4sJb/5UfeVkl/w5vIpQ1b8+5PlxnjyX6XkZ31jl1Lyo756eyn5thHJa4IgCHoBGxZ2bmxvKNwgCAYWHbzC7fVNM0lrS/qNpJmSJudosHf39jyCIBigdHDFh15VuEolcy8DbrK9ie0dSO4WLRmeJcWKPAiC+hhY5OZHH9HbK9y3AK/YPrPSYPsx26dJGizpZEkTJd0t6ShYnB/hZkkTgPvy+Y2SLs+r5O9KOkzSXZLukfTafN+7JN2Zcyn8TdLauX28pPMk3ZDvj2Q3QTBgMHhR86OP6G2FuxUwpc61jwLP2d4R2BE4QtLG+dr2wPG2K25e2wJHA68jBUFsZnsn4Fzg2CxzC7Cz7e1IiSc+XxhrC+DtpAQVX5NUDPsFUi4FSZMkTXrFzYLagiDoCEzaNGt29BF9+ogu6QxgV1J9sseAbQp5JVcDRudrd9l+tHDrRNtP5j4eAa7J7fcAe+TX6wO/k7QusCJQvP+KXHxynqSngLVJySkWU8ylsNqQNTvXCh8EwdLEptliZpBWqwDY/iTwVmBNQMCxtsfkY2PbFUVaHVVWzIWwqHC+iCVfIqcBp9t+PXAUUHTMjFwKQTBQiU2zxVwHDJP08ULbyvnn1cDHK4/3kjaTNHwZxirmUvjQMvQTBEG/oQVl24cKt1dXdrYt6QDgx5I+D/yHtHo9AbiYVOVhSvZm+A9wwDIMNx64WNIzJEW/cWPxIAj6PQaiiOQSsu314DqXv5SPIjdQSK1YzJeQz8fVumb7cuDyGuOPrzrfurWZB0HQL+hgG27YLlvAw1Zk4egSOSrueqZU/z2dG2HQNluUvmf+6iuVG+P+cv0verlnPT/K9j/s6QWl5D13bin5svN5+JAujjNN2fKbzTKWLhubfGtqKfm+WWdGaG8QBEHvYHAf+tk2IxRuEAQDiz6MJGtGJycgb4ikFxpcu6279wZB0M8JL4XeQdIQ2wtsdymhHgTBcoDd0V4K/XaFW6E610JueyH/XFfSTZKmSbpX0m6F+76Vy/fcUcmzEATBAKCDV7j9XuFmqnMtVDgUuNr2GFL+hWm5fThwh+1tgZuAI6o7LOZSmD+/Wfm0IAg6A+OFC5sefcVAMSlU51qoMBE4L0evXWZ7Wm5/Bfhzfj0ZeFv1jcVcCquOWK9zrfBBECyhkp6xQxkoK9yaS1DbNwG7k0J8z5f0wXxpvr34uSJyKQTBQCLSM/YNkjYC/m37HFLqxu2b3BIEQT/GgBe56dEKkvaW9KCkhyV9ocb1oZJ+l6/fKWlUsz4HtMIFxgHTJU0FDgJO6dvpBEHQo7g9CcglDQbOAN4BbAkcImnLKrGPAs/Y3hT4MfC9Zv3220dp2yPyzxso5FaounYBcEG9e/PrS4BLenCqQRD0Im3aFNsJeNj2TABJFwH7kz2hMvuTkmRB0iGnS1LBXNkFNbgWZCT9h5QgvZqRwJwSXZWV740xQj7ke3uMevIb2V6zRD9dkPSX3H8zhgHFBBdn543ySj8HAnvb/lg+/wDwBtvHFGTuzTKz8/kjWabu76LfrnB7k3ofAkmTbI9ttZ+y8r0xRsiHfH+YU6vY3rsn+m0XA92GGwRB0B0eBzYonK/PkoIGXWRyRfHVgP826jQUbhAEQVcmAqMlbSxpRVIO7wlVMhNYUk3mQOC6RvZbCJPCsnJ2c5Flku+NMUI+5Ht7jO7MqVexvUDSMaTSX4OB82zPkHQSMMn2BODnwK8kPQw8Tf3CCouJTbMgCIJeIkwKQRAEvUQo3CAIgl4iFG4QBEEvEQo3CIKglwiFGwRB0EuEwg2CIOglQuEGQRD0Ev8fwmHHRUqmG3sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 在混淆矩阵中记录正确预测\n",
    "confusion = Tensor(np.zeros((n_categories, n_categories)), mstype.float32)\n",
    "n_confusion = 1000\n",
    "\n",
    "# 获取每行的输出\n",
    "def evaluate(line_tensor):\n",
    "    hidden = rnn.initHidden()\n",
    "\n",
    "    for i in range(line_tensor.shape[0]):\n",
    "        output, hidden = rnn(line_tensor[i], hidden)\n",
    "\n",
    "    return output\n",
    "\n",
    "# 运行样本，并记录正确的预测\n",
    "for i in range(n_confusion):\n",
    "    category, line, category_tensor, line_tensor = randomTrainingExample()\n",
    "    output = evaluate(line_tensor)\n",
    "    guess, guess_i = categoryFromOutput(output)\n",
    "    category_i = all_categories.index(category)\n",
    "    confusion[category_i,guess_i] += 1\n",
    "\n",
    "# 标准化输出\n",
    "for i in range(n_categories):\n",
    "    confusion[i] = confusion[i] / np.sum(confusion[i])\n",
    "\n",
    "(!!!拆分出来，加中文注释)\n",
    "\n",
    "    \n",
    "# 绘制图表\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "cax = ax.matshow(confusion.asnumpy())\n",
    "fig.colorbar(cax)\n",
    "\n",
    "# 设定轴\n",
    "ax.set_xticklabels([''] + all_categories, rotation=90)\n",
    "ax.set_yticklabels([''] + all_categories)\n",
    "\n",
    "# 在坐标处添加标签\n",
    "ax.xaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "ax.yaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自定义输入\n",
    "\n",
    "下面同学们可以选择几个名称，通过RNN网络进行预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "> Dovesky\n",
      "(-2.88) Vietnamese\n",
      "(-2.88) Chinese\n",
      "(-2.88) Arabic\n",
      "\n",
      "> Jackson\n",
      "(-2.87) Korean\n",
      "(-2.88) Irish\n",
      "(-2.88) German\n",
      "\n",
      "> Satoshi\n",
      "(-2.88) Spanish\n",
      "(-2.88) Greek\n",
      "(-2.88) Japanese\n"
     ]
    }
   ],
   "source": [
    "def predict(input_line, n_predictions=3):\n",
    "    print('\\n> %s' % input_line)\n",
    "    \n",
    "    output = evaluate(lineToTensor(input_line))\n",
    "\n",
    "    # Get top N categories\n",
    "    topk = ops.TopK(sorted=True)\n",
    "    top_n, top_i = topk(output,3)\n",
    "    predictions = []\n",
    "\n",
    "    for i in range(n_predictions):\n",
    "        value = top_n.asnumpy()[0][i].item()\n",
    "        category_index = top_i.asnumpy()[0][i].item()\n",
    "        print('(%.2f) %s' % (value, all_categories[category_index]))\n",
    "        predictions.append([value, all_categories[category_index]])\n",
    "\n",
    "        \n",
    "(!!!全部都是一样的参数，你让我选择什么？)\n",
    "predict('Dovesky')\n",
    "predict('Jackson')\n",
    "predict('Satoshi')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
